With compressing technologies such as MPEG (Moving Picture Experts Group) system that have been recently used for digital broadcasts, many programs can be broadcast through a limited transmission medium (a wireless transmission medium or a wired transmission medium). Likewise, when a broadcast provider transmits programs, the rental fee of a transponder of a satellite circuit is expensive. Thus, from a view point of cost, it is effective to compress an occupied band-width. This situation is the same with a material transmission using a ground wave or a commercial wire circuit. Thus, when data is transmitted from a site to a broadcasting station or data is transmitted between broadcasting stations, it is meaningful to use an MPEG stream.
A major benefit for applying a compressing technology such as MPEG system to a picture material in a broadcasting station is to reduce the storage capacity of a picture material archiver/server. When a non-linear editing operation was not required, a picture material was archived on a tape whose running cost is low. However, since the non-linear editing operation has been required in recent years, it is necessary to reduce the storage capacity of a non-linear record medium (hard disk, DVD, or the like).
Next, with reference to FIG. 15, an outline of a transmission system corresponding to MPEG standard will be described. The transmission system has an encoder portion 110 and a decoder portion 120. The encoder portion 110 has a video data portion and an audio data portion. The video data portion comprises a video encoder 111 and a packetizer 112. The video encoder 111 encodes input video data DV and outputs a video elementary stream ES. The packetizer 112 packetizes the video elementary stream ES received from the video encoder 111, adds a header and so forth thereto, and outputs a video packetized elementary stream PES. The audio data portion of the encoder portion 110 comprises an audio encoder 113 and a packetizer 114. The audio encoder 113 encodes input audio data DA and outputs an audio elementary stream ES. The packetizer 114 packetizes the audio elementary stream ES received from the audio encoder 113, adds a header and so forth thereto, and outputs a video packetized elementary stream PES. The encoder portion 110 also has a multiplexer 115. The multiplexer 115 multiplexes the elementary streams received from the packetizers 112 and 114, generates transport stream packets (each of which is composed of 188 bytes), and outputs them as a transport stream TS.
The decoder portion 120 of the transmission system shown in FIG. 15 comprises a demultiplexer 121, depacketizers 122 and 124, a video decoder 123, and an audio decoder 125. The demultiplexer 121 demultiplexes the transport stream received through a transmission medium 116 into a video PES and an audio PES. The depacketizers 122 and 124 depacketize the video PES and the audio PES, respectively. The video decoder 123 decodes the video ES received from the depacketizer 122. The audio decoder 125 decodes the audio ES received from the depacketizer 124. The video decoder 123 outputs a base band video signal DV. The audio decoder 125 outputs a base band audio signal DA. The decoder portion 120 is referred to as IRD (Integrated Receiver/Decoder).
Concentrating on video data, the operation of the system shown in FIG. 15 will be described. In the encoder portion 110, the video encoder 111 encodes an input video data DV of which each picture has the same bit quantity, converts each picture to a bit quantity corresponding to its redundancy, and outputs the resultant data as a video elementary stream. The packetizer 112 absorbs (averages) the fluctuation of the bit quantity of the video elementary stream on the time base and outputs the resultant data as a video packetized elementary stream. The transport stream multiplexer 115 multiplexes the video packetized elementary stream received from the packetizer 112 and the audio packetized elementary stream received from the packetizer 114, generates the multiplexed data as transport stream packets, and supplies them as a transport stream TS to the decoder portion 120 through the transmission medium 116.
In the decoder portion 120, the transport stream demultiplexer 121 demultiplexes a transport stream into a video packetized elementary stream and an audio packetized elementary stream. The depacketizer 122 depacketizes the video packetized elementary stream and outputs the resultant data as a video elementary stream. The video decoder 123 decodes the video elementary stream and outputs the resultant data as video data DV.
The decoder portion 120 performs a variable bit quantity extracting process for each reproduced picture from a received stream with a fixed bit rate using for example a 1.75 Mbit VBV (Video Buffering Verifier) buffer. Thus, the encoder portion 110 should control a generated bit quantity of each picture so as to prevent the VBV buffer from overflowing or underflowing. Such a controlling process is referred to as VBV buffer process.
As described above, from view points of effective use of limited transmission resources for a multi-channel broadcasting system and a reduction of the running-cost of a transmission circuit, it is very attractive to use an encoded stream. However, an MPEG stream corresponding to a highly efficient compressing system has restrictions against broadcast materials.
The compressing technology of the MPEG system has the following features: (1) In the MPEG system, an encoding process is performed corresponding to frame correlation of each GOP (Group Of Picture). (2) Each picture that has been encoded corresponding to the MPEG format has a variable bit length. (3) In MPEG2 format, the bit rate is controlled (a process of the VBV buffer) so that the buffer condition of the destination IRD is satisfied. (4) When an encoding process corresponding to the MPEG2 format is performed for transmitting data, the bit rate of the data is controlled corresponding to the capacity of a transmission path.
Due to the features of the MPEG system, when an MPEG bit stream is received and edited, the following problems will take place. In other words, when data is edited for each frame, after an MPEG stream is decoded to a base band signal, the data should be edited. Thereafter, the resultant base band signal should be encoded into an MPEG stream. Thus, whenever an editing operation including a switching operation is performed, the encoding process and the decoding process are repeated. Normally, when a decoding-encoding chain of a base band signal to an MPEG stream is performed, the resultant picture largely deteriorates. When an encoded bit stream is switched at any position, even if it is at the end of an encoding unit (in reality, even in a closed GOP structure that does not use the correlation of GOPs), the continuity of the buffer controlling operation is lost. Thus, the restrictions of the VBV buffer cannot be satisfied. As a result, a decoded picture freezes or breaks because the buffer overflows or underflow.
Due to such problems, it was said that it is substantially impossible to edit data as an MPEG stream. Thus, even if a compressed multi-channel broadcast corresponding to MPEG system is performed, a means for editing a base band material and finally encoding the edited data into an MPEG stream is used. When an original material is an MPEG stream, after the base band signal is decoded into a base band signal, it is edited by a conventional base band editing device. Thus, after the editing operation is performed, the resultant picture quality remarkably deteriorates. In addition, when a special effect such as gain adjustment is performed or when a logo of a broadcasting station is inserted, the MPEG stream cannot be used.
Next, such problems will be practically described with several examples of an editing system in a broadcasting station. FIG. 16 shows master servers and an interface of an editing studio. Each of the master servers has an MPEG stream archiver. In the broadcasting station, data is sent as a base band signal. In FIG. 16, reference numeral 101 is a master archiver/server in a broadcasting station. The archiver/server 101 is a non-linear archiver that has a storing portion. The storing portion stores a material of an MPEG compressed stream so as to reduce the data amount of the material. Both the archiver and the server store picture materials. However, the archiver is a device that dedicatedly stores a picture material. In contrast, the server outputs a picture material corresponding to a request received from an external device. According to the present invention, since both the archiver and the server have a function as a picture storing portion, the present invention can be applied to both the archiver and the server. Thus, in the specification, a term archiver/server is used.
The archiver/server 101 has an MPEG decoder that decodes an MPEG stream received from the storing portion. Base band video data S1 and S2 generated by an MPEG decoder are input to the editing studio 102. A transmission protocol of transmission paths in the broadcasting station is based on a base band signal. The editing studio 102 performs an editing operation for connecting the video data S1 and S2 (splice editing operation, AB roll operation, or the like). Video data S3 (a base band signal) that has been edited is input to the archiver/server 103. The archiver/server 103 has an MPEG encoder. The MPEG encoder supplies the edited result as an MPEG stream to a storing portion.
FIG. 17 shows an example of the structure of the editing studio 102. Since the data amount of video data of a base band signal is large (namely, the bit rate thereof is high), a tape medium is used as a record medium. In other words, video data S1 is recorded to a linear storage 104a. Video data S2 is recorded to a linear storage 104b. The linear storages 104a and 104b function as players. The video data Sa and Sb are supplied to an editor and switcher 105. Video data Sc as an edited result of the editor and switcher 105 is recorded to a linear storage 104c that functions as a recorder. The linear storage 104c outputs edited data as video data S3.
As shown in FIG. 18, the editing studio 102 may be composed of non-linear storages 106a, 106b, and 106c that use non-linear record mediums (hard disks, optical discs, and so forth). However, when a base band signal is handled with a non-linear record medium, since the data amount of the base band signal is large, the record medium is expensive. Thus, the structure of which a linear storage is disposed to each editing studio is not practical. In the editing system shown in FIG. 16, whenever an editing operation is performed, a decoding-encoding chain takes place. Thus, the picture quality of a material cumulatively deteriorates.
FIG. 19 shows master servers and an interface of an editing studio in the case that a transmission protocol of transmission paths in a broadcasting station is an MPEG stream. An archiver/server 131 and an archiver/server 133 store materials as MPEG streams. The archiver/server 131 outputs an MPEG stream to an editing studio 132. The archiver/server 131 receives an MPEG stream from the editing studio 132. Thus, the archiver/server 131 and the archiver/server 133 do not have an MPEG decoder and an MPEG encoder. With an MPEG stream, two or more picture materials can be multiplexed as streams TS1 and TS2. In such a multi-channel system, transmission paths can be effectively used. The streams TS1 and TS2 may be elementary streams or transport streams.
FIGS. 20 and 21 show a first example and a second example of the editing studio 132 of the system shown in FIG. 19, respectively. In the first example shown in FIG. 20, a stream TS1 is decoded into streams TS1a and TS1b. MPEG decoders 134a and 134b convert the streams TS1a and TS1b into respective base band signals. The resultant base band signals are stored to linear storages 135a and 135b. Base band video data Sa and Sb obtained by the linear storages 135a and 135b that function as players are supplied to a base band editor and switcher 136. The base band editor and switcher 136 supplies the edited result as video data Sc to a linear storage 135c that functions as a recorder. Video data received from the linear storage 135c is supplied to an MPEG encoder 134c. The MPEG encoder 134c outputs encoded data as an MPEG stream TS2.
In the second example of the editing studio 132 shown in FIG. 21, non-linear storages 137a, 137b, and 137c are used instead of the linear storages 135a, 135b, and 135c, respectively. In the example shown in FIG. 21, an MPEG stream can be sent through a transmission path of a broadcasting station so that a multi-channel system can be easily structured. However, in the first and second examples shown in FIGS. 20 and 21, whenever an editing operation is performed, a decoding-encoding chain takes place. Thus, the picture quality of the material unignorably deteriorates. In addition, the picture quality cumulatively deteriorates. Moreover, when a base band signal is handled with a non-linear record medium, since the data amount of the base band signal is large and the non-linear record medium is expensive, the second example shown in FIG. 21 of which a non-linear record medium is disposed in each broadcasting station is not practical.
To prevent a material from deteriorating against a decoding-encoding chain, a material is archived as a base band material. In this case, since the data amount of a picture material becomes large, it is difficult to store it to a non-linear record medium.
As a means for solving problems of material deterioration and record capacity, it is preferred to edit data as a stream. However, to do that, there are problems due to the features of the MPEG stream. To solve these problems, there are several methods. For a problem of overflow, the number of bits of each picture is counted. The VBV buffer is simulated so as to insert dummy data thereto. However, for a problem of underflow, no solving method is known. In this case, a picture freezes.
On the other hand, a bit rate controlling method is known. In this method, before an encoding process is performed, a switching point is designated so that a predetermined buffer amount takes place at the switching point. In this method, the problem of VBV buffer will be solved. However, the problem is solved at only the predetermined switching point. Thus, the scope of this method is limited.
In addition, to solve a problem of deterioration of picture quality due to a decoding-encoding chain, when a stream is decoded, information necessary for an encoding process and a decoding process is extracted and multiplexed with a base band signal. The information is referred to as codec information. When a re-encoding process is performed, with the codec information, the accuracy of reconstruction of a picture is improved. This process is referred to as trans codec process. The codes information contains information of moving vector, quantizing step, picture type, and so forth.
The information amount of codec information is not small. Thus, since the base band signal does not have a sufficient auxiliary region in which the codec information is multiplexed, the remaining codec information should be multiplexed with a valid picture region or transmitted through another line.
FIG. 22 shows an example of the structure of which an editing studio is structured with a transcoding process. The transcoding process is performed so as to prevent the picture quality of a material from cumulatively deteriorating against a decoding-encoding chain corresponding to each editing operation. In FIG. 22, codec information is sent through a path different from a material signal line. MPEG decoders 134a and 134b convert streams TS1a and TS1b into respective base band signals as base band video data Sa and Sb. The base band video data Sa and Sb are supplied to a base band editor and switcher 136. The base band editor and switcher 136 supplies the edited result as video data Sc to an MPEG encoder 134c. The MPEG encoder 134c re-encodes the video data Sc and outputs the re-encoded data as an MPEG stream TS2.
The editing studio shown in FIG. 22 also has information detectors 141a and 141b, signal lines 142a, 142b, and 142c, an information estimator 144, and a codec information adaptor 143. The information detectors 141a and 141b detect codec information used in the MPEG decoders 134a and 134b from streams or the decoders 134a and 134b, respectively. The signal lines 142a, 142b, and 142c transmit codec information. The information estimator 144 allows the encoder 134c to use the codec information. The codec information adaptor 143 organically connects the codec information with the edit information of the base band editor and switcher 136.
When codec information is sent through another line, the editor and switcher 136 performs an editing operation. To handle codec information sent through another system, a special structure such as the codec information adaptor 143 should be added. In other words, a conventional editing studio that handles a base band signal cannot be used.
FIG. 23 shows the structure of an editing studio that allows such a problem to be solved. In other words, in the structure shown in FIG. 23, codec information is multiplexed with a valid region of a base band signal. The editing studio shown in FIG. 23 has information detectors 141a and 141b that detect codec information from input streams TS1a and TS1b or decoders 134a and 134b, respectively. Imposers 145a and 145b multiplex the detected codec information with video data Sa and Sb as the base band signals, respectively. The multiplexed base band signals are supplied to a base band editor and switcher 136. As an example of the multiplexing method, codec information is randomly multiplexed as the least significant bit of each sample of video data.
The base band editor and switcher 136 outputs video data in which codec information has been multiplexed. The video data is supplied to a separator 146. The separator 146 separates codec information from the video data received from the base band editor and switcher 136. The video data Sc separated by the separator 146 is supplied to an MPEG encoder 134c. The MPEG encoder 134c re-encodes the video data Sc using the codec information received from the separator 146.
FIG. 24 shows the structure of which non-linear storages 147 and 148 are added to the structure shown in FIG. 23. The non-linear storage 147 outputs a stream that has been recorded and reproduced to MPEG decoders 134a and 134b. The non-linear storage 148 records a stream that is re-encoded by an MPEG encoder 134c. 
As shown in FIGS. 23 and 24, when codec information is multiplexed with a base band signal and then the multiplexed signal is transmitted, the base band editor and switcher 136 does not need a special device such as a codec information adaptor. However, in the method of which codec information is inserted into a valid region of a picture signal, even if the codec information is converted into random data and then multiplexed with a picture signal, a picture distorts and the S/N ratio thereof deteriorates.
In the structures shown in FIGS. 23 and 24, when codec information is multiplexed with a base band signal, a multiplexing means is disposed in an editing studio. FIG. 25 shows an example of the structure of which a means for multiplexing and demultiplexing codec information is disposed in an archiver/server. Referring to FIG. 25, a archiver/server 151 comprises MPEG decoders 155a and 156a, information detectors 155b and 156b, and imposers 157a and 157b. The MPEG decoders 155a and 156a decode an MPEG stream received from a storing portion 154. The information detectors 155b and 156b detect codec information from respective streams. The imposers 157a and 157b multiplex codec information with video data as base band signals.
Video data S11 and S12 in which codec information has been multiplexed are supplied to an editing studio 152. The editing studio 152 handles a base band signal. As with the structure shown in FIG. 24, the editing studio 152 is composed of a linear storage and a base band editor and switcher.
Video data S13 as a base band signal in which codec information received from the editing studio 152 has been multiplexed is supplied to an archiver/server 153 that stores video data as an edited result. A separator 158 separates codec information from the video data S13. An MPEG encoder 159 re-encodes the resultant video data using the codec information. A stream received from the MPEG encoder 159 is stored to a storing portion 160. However, actually, the structure shown in FIG. 25 does not correctly function. In other words, connections in the structure shown in FIG. 25 are invalid. In the editing studio 152, video data is recorded to a conventional record medium such as a VTR (Video Tape Recorder) that records a base band signal. Of course, the conventional VTR does not support a function for extracting codec information and supplying the extracted codec information to the next stage. Moreover, since most conventional digital VTRs use a compressing system other than the MPEG system, information multiplexed in a valid region of a signal is compressed and decompressed in the same manner as video data. Thus, since codec information is compressed and decompressed, the resultant video data distorts. Consequently, the codec information cannot be used. Even if codec information is superimposed at the least significant bit of video data, the least significant bit is varied by the compressing process and the decompressing process of the VTR.
On the other hand, in the structures shown in FIGS. 23 and 24, a stream is transmitted. Additional structural elements such as an MPEG decoder and a re-encoder are disposed in an editing studio. Thus, a probability for interfacing a conventional VTR with a base band signal in which codec information has been multiplexed is excluded. However, as described above, when codec information is inserted into a valid region of a picture signal, the resultant picture distorts and the S/N ration thereof deteriorates.
Therefore, an object of the present invention is to provide an editing system, an editing controlling apparatus, and an editing controlling method that allow a storage medium and a transmission medium to be effectively used, the picture quality to be suppressed from deteriorating, and a conventional base band editing device to be used.
Another object of the present invention is to provide an editing system, an editing controlling apparatus, and an editing controlling method that allow an edit position to be detected without need to obtaining edit position information from an editing device.
A further object of the present invention is to provide an editing system, an editing controlling apparatus, and an editing controlling method that allow codec information for a re-encoding process to be used in a unit smaller than a picture and the picture quality of a re-encoded picture to be prevented from deteriorating.